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QUALITY  msPECTED  1 


PREFACE 


The  purpose  of  this  document  is  to  complement  the  activities  of  the 
Air  Force  and  others  as  they  examine  key  joint  campaign  modeling 
issues,  capabilities,  and  requirements  for  inclusion  into  new  “next 
generation”  campaign  model  development  efforts.  The  issues  out¬ 
lined  are  believed  to  be  of  such  importance  that  the  successful 
development  of  a  new  model  (or,  for  that  matter,  the  enhancement 
or  derivation  of  an  existing  model)  may  well  hinge  on  the  resolution 
of  these  challenges. 

This  document  is  a  companion  to  the  longer  report  entitled  Modeling 
for  Campaign  Analysis:  Lessons  for  the  Next  Generation  of  Models, 
MR-71 1-AF,  which  provides  a  detailed  review  of  the  issues  discussed 
herein  and  discusses  options  for  their  resolution.  That  report  is 
organized  as  a  reference  so  that  the  reader  interested  in  detailed  dis¬ 
cussions  of  particular  issues,  such  as  those  associated  with 
databases,  model  structure,  and  model  development,  can  go  directly 
to  the  relevant  sections. 

This  research  was  conducted  as  part  of  our  work  in  modeling  and 
simulation  improvement  in  Project  AIR  FORCE  and  was  sponsored 
by  tbe  Director  of  Modeling,  Simulation,  and  Analysis  (XOM).  Tbe 
report  should  be  of  interest  to  the  variety  of  modelers,  analysts,  and 
decisionmakers  who  use  models  as  part  of  the  campaign  analysis 
process,  especially  those  involved  in  defining  requirements  for  new 
campaign  models. 
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RELATED  DOCUMENTATION 

We  have  produced  five  reports  and  a  research  brief  as  part  of  this 
work.  This  executive  summary  provides  an  overview  of  important 
issues  and  approaches  for  campaign  models  designed  for  analysis. 
Still  to  come  is  the  full  report,  which  will  address  those  issues  in 
detail,  describe  alternative  approaches,  and  discuss  how  a  campaign 
model  should  be  used  in  the  analysis  process.  A  third  report.  The 
Theater-Level  Campaign  Model:  A  Research  Prototype  for  a  New 
Generation,  MR-388-AF,  forthcoming,  provides  a  more  in-depth 
description  of  the  important  models  we  implemented  in  the  Theater- 
Level  Campaign  model.  The  MAPVJEW  Users  Guide,  MR-160-AF/A, 
1993,  describes  a  graphical  user  interface  developed  as  part  of  the 
theater-level  campaign  work  specifically  for  the  generalized  network 
gameboard  that  underlies  the  Theater-Level  Campaign  model.  An 
earlier  workshop  report.  New  Issues  and  Tools  for  Future  Military 
Analysis:  A  Workshop  Summary,  N-3403-DARPA/AF/A,  1992,  de¬ 
scribes  conclusions  reached  at  the  start  of  this  project  about  the  need 
for  new  models  for  analysis  in  the  post-Cold  War  era. 

PROJECT  AIR  FORCE 

Project  AIR  FORCE,  a  division  of  RAND,  is  the  Air  Force  federally 
funded  research  and  development  center  (FFRDC)  for  studies  and 
analyses.  It  provides  the  Air  Force  with  independent  analyses  of 
policy  alternatives  affecting  the  development,  employment,  combat 
readiness,  and  support  of  current  and  future  aerospace  forces. 
Research  is  being  performed  in  three  programs:  Strategy  and 
Doctrine,  Force  Modernization  and  Employment,  and  Resource 
Management  and  System  Acquisition. 

In  1996,  Project  AIR  EORCE  is  celebrating  50  years  of  service  to  the 
United  States  Air  Force.  Project  AIR  FORCE  began  in  March  1946  as 
Project  RAND  at  Douglas  Aircraft  Company,  under  contract  to  the 
Army  Air  Eorces.  Two  years  later,  the  project  became  the  foundation 
of  a  new,  private  nonprofit  institution  to  improve  public  policy 
through  research  and  analysis  for  the  public  welfare  and  security  of 
the  United  States — what  is  known  today  as  RAND. 
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Chapter  One 

INTRODUCTION 


This  report  summarizes  our  experiences  and  observations  in  devel¬ 
oping  and  applying  models  to  support  campaign  analyses.  Our  goal 
is  to  provide  key  lessons  learned  to  those  involved  in  ongoing  model¬ 
ing  efforts  to  move  the  technology  of  campaign  modeling  into  the 
next  generation.  To  this  end,  our  primary  purpose  is  to  focus  these 
efforts  on  improving  analytic  capability  rather  than  on  advancing 
computer  science  or  on  satisfying  the  ever-present  demands  for 
more  and  more  detail  in  models.  Our  desire  is  to  link  the  needs  of 
analysis  to  the  content  and  use  of  a  campaign  model,  to  describe 
some  of  the  important  new  (and  old)  challenges  for  the  next  genera¬ 
tion  of  models,  and  to  discuss  promising  (and  not  so  promising) 
approaches. 

THE  PROBLEM 

The  U.S.  military’s  force  structure  and  defense  strategy  are  increas¬ 
ingly  affected  by  the  use  of  computer  models.  New  technologies 
promise  to  facilitate  even  broader  use  of  models  in  the  Department 
of  Defense  through  advanced  graphics,  networked  simulations,  and 
much  faster  computation.  For  example,  simulations  netted  with  real 
exercises  fighting  against  virtual  and  live  forces  in  widely  distributed 
geographic  locations  have  already  been  demonstrated.^  The  promise 


^Such  demonstrations  include  the  Synthetic  Theater  of  War-Europe  (STOW-E)  as  part 
of  the  Atlantic  Resolve  (formerly  Reforger)  exercise,  the  Ballistic  Missile  Defense 
Organization  (BMDO)  Technical  Engineering  Demonstration  (TED),  DARPA’s 
Warbreaker,  and  the  Army’s  Anti-Armor  Advanced  Technical  Demonstrations 
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of  lower  exercise  costs,  of  reducing  (or  perhaps  eliminating)  some 
high-cost  components  of  weapon  tests,  and  of  using  more  realistic 
models  for  analysis  continues  to  drive  large  investments  in  modeling 
and  simulation  technology  and  software.^ 

Despite  these  investments  and  the  impressive  technological 
demonstrations,  many  aspects  of  the  current  state  of  defense 
modeling  are  disturbing.  The  models  in  use,  largely  developed  for 
NATO-Warsaw  Pact  Campaign  Assessment  calculations  during  the 
Cold  War,  do  not  reflect  some  fundamental  changes  in  the  nature  of 
warfare:  the  effects  of  modern  systems,  particularly  sensors;  and 
command  and  control.  Individual  military  services  are  concerned 
that  the  models  do  not  adequately  and  fairly  represent  the 
contributions  of  those  services  in  a  joint  warfare  arena.  Many  believe 
that  the  national  security  environment  has  changed  so  dramatically 
in  terms  of  uncertainty  about  threats,  declining  defense  outlays,  and 
the  needs  for  analysis  that  the  fundamental  nature  of  the  models  in 
use  for  defense  analysis  must  change.  Political  and  military  leaders 
are  confused  by  the  proliferation  of  models  and  model-based  study 
results,  which  often  provide  conflicting  conclusions  and  diverse 
information.  Furthermore,  decisionmakers  frequently  desire  rapid 
responsiveness  that  current-generation  campaign  models  and  their 
users  often  cannot  deliver.^ 

In  a  February  1995  memo  to  the  Director,  Program  Analysis  and 
Evaluation,  then-Deputy  Secretary  of  Defense  John  Deutch  said, 

loint  theater  models  play  an  important  role  in  assessing  the  capabil¬ 
ities  of  our  forces  and  programs  to  execute  our  strategy  and  in  mea¬ 
suring  the  impact  of  changes  in  the  defense  program  on  warfighting 
capabilities.  As  we  discussed  at  the  recent  review  of  the  mobility 


(A2ATD).  (U.S.  Department  of  the  Army,  1994;  M1A2  SIMNET-D  Synthetic  En¬ 
vironment  Experiment  Committee,  1993;  Johnson,  1994.) 

^For  a  description  of  the  distributed  simulation  plans  see  DIS  Steering  Committee 
(1994).  It  is  estimated  that  DARPA  spends  $130  million  annually  on  Advanced 
Distributed  Simulation  activities,  including  Warbreaker  and  STOW.  The  Army  is  esti¬ 
mated  to  spend  $300  million  in  DlS-related  areas. 

^It  should  be  noted  here  that  responsiveness  and  the  ability  to  analyze  a  complex, 
campaign-level  issue  in  some  breadth  and  depth,  rather  than  the  abiiity  to  run  a 
model  rapidly,  are  really  the  sources  of  tension.  A  very  real  tradeoff  exists,  sometimes 
not  appreciated  by  decisionmakers,  between  the  need  for  responsiveness  and  the 
quality  of  the  answer. 
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requirements  update  study,  many  of  our  analytical  efforts  have 
highlighted  the  limitations  of  the  modeling  tools  currently  available. 
Those  models  are  grounded  in  Cold  War  theory  about  the  use  and 
deployment  of  forces  and  the  nature  of  combat  operations.  They 
have  only  limited  capability  to  address  key  issues  from  an  inte¬ 
grated  joint  operational  perspective.  They  are  also  unable  to  mea¬ 
sure  adequately  the  value  of  the  Department’s  investments  in 
reconnaissance,  surveillance,  and  intelligence  and  new  weapon  sys¬ 
tems.  Nor  do  they  adequately  represent  the  impact  of  such  factors 
as  readiness  and  training,  logistics,  or  weapons  of  mass  destruction. 
Moreover,  the  realism  of  the  basic  methodologies  that  drive  the 
models’  results  is  in  need  of  review.  Therefore,  as  a  manner  of 
immediate  concern,  it  is  essential  to  upgrade  and  refine  our  current 
models  and  to  begin  development  of  a  new  generation  of  models 
that  the  Department  will  need  to  address  critical  joint  warfare  issues 
effectively  in  the  future.^ 

The  current  reliance  on  quantitative  analysis  and  models  has 
increased,  while  the  downward  pressure  on  defense  budgets  has  put 
funding  of  modeling,  simulation,  and  analysis  activities  in  question. 
As  decisionmakers  grapple  with  this  problem,  there  is  a  trend  to  try 
to  consolidate  and  improve  the  models  once  and  for  all,  as  though  it 
is  possible  to  get  them  “right”  and  thus  remedy  the  situation. 

SOME  INITIAL  DISTINCTIONS:  CAMPAIGN  ANALYSIS  AND 
THE  CAMPAIGN  MODEL 

The  focus  of  this  discussion  is  on  campaign  analysis,  although  much 
of  what  we  have  to  say  applies  more  generally.^  The  importance  of 
campaign  analysis  for  defense  problems  is  that  it  embraces  the  big 
picture  in  terms  of  the  total  forces  involved,  including  the  joint 
actions  of  air,  ground,  and  naval  forces,  as  well  as  the  play  of  coali¬ 
tion  forces.  Campaign  analysis  considers  many  of  the  most  impor- 


^As  a  result  of  this  memo  to  PA&E,  a  four-phased  initiative  was  begun.  Phase  one 
specifically  deals  with  the  near-term  enhancement  of  an  existing  model  (TACWAR). 
Phases  two  and  three  focus  on  future  models  and  modeling  environments  for  the 
midterm  (based  on  a  joint  model  development)  and  long-term  (based  on  the  high- 
level  architecture).  Phase  four  cuts  across  all  these  activities  to  provide  field  support 
of  the  models. 

^For  convenience,  we  often  drop  the  “campaign”  modifier  and  use  only  the  term 
“analysis.” 
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tant  determinants  of  the  outcome  of  a  war,  such  as  territory  lost, 
achievement  of  air  superiority,  and  overall  attrition.  At  the  campaign 
level  one  can  start  to  gain  insight  into  “how  much  is  enough”  military 
force,  because  all  forces  are  included. 

Campaign  analysis  is  also  used  to  study  the  interactions  of  strategy, 
force  allocation,  and  system  capabilities.  For  example,  the  addition 
of  a  more-effective  surfece-to-air  capability  may  permit  an  allocation 
of  more  combat  aircraft  to  attack  enemy  ground  forces  rather  than 
using  them  for  defense.  Typically,  these  models  have  represented 
territorial  conquests  or  destruction  of  enemy  targets  but  could  also 
include  naval  exchanges  and  blockades  or  some  aspects  of  informa¬ 
tion  warfare.  Cumulative  effects  over  time  are  important  in  cam¬ 
paign  analysis,  and  multiday  strategies,  objectives,  and  outcomes 
must  be  considered.  Finally,  it  is  in  campaign  analysis  that  many 
important  scenario  factors  play,  such  as  timing  of  the  deployment  of 
forces,  availability  of  ports  and  air  bases,  nearness  of  the  battle  to 
ocean  access,  and  the  strategy  and  operations  of  the  enemy. 

Models  for  defense  analysis  range  from  engineering  models  of  spe¬ 
cific  subsystems  (a  model  of  an  aircraft  radar  system,  for  example)  to 
engagement  models  (a  surface-to-air  system  engaging  aircraft)  to 
mission  models  (a  model  of  a  complete  flight  of  aircraft  from  takeoff 
to  attack  on  a  target  and  back  again)  to  the  campaign  model,  which 
represents  a  set  of  missions,  operations,  or  battles  in  a  military  cam¬ 
paign.®  Campaign  models  are  also  sometimes  referred  to  as 
“theater”  models  because  their  focus  and  scope  are  usually  associ¬ 
ated  with  a  full  theater  of  conflict  even  though  there  might  be  more 
than  one  campaign  associated  with  a  theater. 

Another  distinction  in  combat  models  is  between  the  tactical,  opera¬ 
tional,  and  strategic  levels  of  conflict.  We  will  use  the  term 
“campaign  model"  to  encompass  the  operational  level  of  conflict, 
fully  understanding  that  this  represents  a  somewhat  different  scope 
in  concepts  and  analysis  from  the  campaign  or  theater  level.  Many 
of  the  points  we  discuss  equally  apply  across  the  broad  range  of 
modeling  and  analysis. 


®Some  campaign  models  allow  simultaneous  analysis  of  multiple  theater  conflicts. 
RAND’s  Joint  Integrated  Contingency  Model  (JICM)  has  such  a  capability,  as  described 
in  Bruce  Bennett  et  al.,  1994. 
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RESEARCH  FOCUS 

This  research  concentrates  on  what  we  believe  to  be  some  of  the 
most  important  problems  and  issues  in  campaign-level  modeling 
and  analysis.  Although  our  focus  is  on  campaign-level  modeling  as  it 
is  developed  and  applied  to  analysis,  many  of  these  issues  are  appli¬ 
cable  to  other  types  of  models  and  for  model  uses  other  than  analy¬ 
sis.  We  believe  that  overcoming  the  difficulties  raised  here  is 
essential  to  future  progress  in  model-based  defense  analysis,  and 
attempting  to  develop  new  models  without  their  resolution  will  lead 
to  only  marginally  better  ways  of  doing  things  we  already  know  how 
to  do.  Furthermore,  it  is  most  important  that  these  issues  be 
addressed  very  early  in  the  development  of  the  model — during  the 
conceptual  design  stage. 

This  report  does  not  offer  a  comprehensive  solution  to  the  problems 
of  campaign  analysis  and  modeling.  Instead,  it  sets  out  the  six  hard¬ 
est  things  we  confronted  in  building  and  applying  our  own  campaign 
model.  ^  We  have  considered  the  relatively  modest  progress  made  in 
campaign  modeling,  relative  to  the  nearly  half-century  of  effort 
devoted  to  it,  and  determined  that  this  is  due  in  part  to  the  lack  of 
accessible  documentation  not  only  on  the  models  themselves  but  on 
the  lessons  learned  from  both  successful  and  failed  analysis  and 
modeling  efforts.  We  hope  to  share  with  the  community  the  lessons 
we  have  learned  so  that  our  successes  can  be  exploited  instead  of 
relearned  and  so  that  our  failures  can  be  avoided  instead  of  recom¬ 
mitted. 

We  have  selected  this  list  because  we  believe  that  these  are  clear 
areas  for  which  positive  steps  can  be  taken,  and  because  we  have 
something  specific  to  contribute.  We  selected  six  as  a  small  number 


^As  part  of  research  on  military  campaigns  of  the  future  by  RAND's  Project  AIR  FORCE 
and  Arroyo  Center,  we  have  designed  and  coded  the  prototype  Theater  Level 
Campaign  (TLQ  model  for  the  investigation  of  joint  force  issues  in  the  post-Cold  War 
era  (see  Hillestad  and  Moore,  forthcoming).  For  this  model  we  investigated  advanced 
features  for  representing  maneuver  in  “nonlinear”  battlefields;  structures  for  better 
simulation  of  command,  control,  and  information  systems;  advanced  setup  and 
graphic  user  interfaces;  and  recent  state-of-the-art,  object-oriented  simulation  soft¬ 
ware  CCACI  Products  Company,  1994).  While  this  document  is  not  specifically  about 
TLC  or  any  other  model,  it  does  draw  from  our  experience  in  many  of  the  most  diffi¬ 
cult  or  controversial  aspects  of  modeling  to  support  campaign  analysis. 
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of  issues  that  can  be  concentrated  on  that  still  covers  a  range  of  the 
thorniest  issues  that  must  be  resolved.^  These  are: 

1.  applying  a  campaign  model  to  analysis 

2.  managing  the  development  process^ 

3.  overcoming  problems  of  data  and  integrating  with  other  models 

4.  creating  flexible  underlying  structures 

5.  representing  decisionmaking  and  command,  control,  communi¬ 
cations,  computers,  and  intelligence  (C^l) 

6.  achieving  transparency  in  the  use  of  a  campaign  model. 

Chapter  Two  of  this  report  describes  these  six  areas  in  more  detail. 
Chapter  Three  proposes  a  plan  for  the  future  for  resolving  some  of 
these  difficulties. 


®Our  list  of  the  six  hardest  things  is  unlikely  to  match  the  reader’s  exactly. 
Furthermore,  our  perspectives  on  these  issues  may  well  differ.  We  encourage  readers 
to  document  their  own  lists  of  “hardest  things,”  for  which  they  will  have  superior 
experience  and  ideas,  and  contribute  the  results  to  the  overall  benefit  of  the  commu¬ 
nity. 

®This  could  be  either  for  a  new  effort  or  for  enhancements  to  an  existing  model. 


Chapter  Two 

APPLYING  A  CAMPAIGN  MODEL  TO  ANALYSIS 


Using  a  campaign  model  for  analysis  requires  maintaining  the 
proper  relationship  between  the  model  and  the  analysis.  What 
seems  to  have  been  lost  in  many  of  the  modeling  developments  is 
that  analysis  is  a  “process”  and  that  the  model  is  but  one  tool  in  that 
process.  A  single  computer  run  of  a  combat  model  serves  almost  no 
purpose  at  all,  because  it  is  merely  a  hypothesis  about  how  things 
might  come  out  and  provides  at  best  a  very  weak  prediction  of  what 
might  happen.  Far  too  often,  unanalyzed  model  results  are  tran¬ 
scribed  onto  transparencies.  The  model  is  only  one  of  many  compo¬ 
nents  of  the  analytic  process,  which  includes 

•  formulating  the  problem 

•  specifying  objectives,  constraints,  and  measures 

•  obtaining  and  manipulating  data 

•  generating,  evaluating,  and  prioritizing  alternatives 

•  developing  and  testing  hypotheses. 

When  analysis  is  performed  well,  for  example,  unknowns  and  uncer¬ 
tainties  are  managed,  and  recommendations  are  developed  that 
account  for  these  complexities  and  their  related  risks.  Analysis  and 
modeling,  particularly  modeling  enhancements,  focus  on  big-picture 
issues,  such  as  defense  strategy  in  a  multipolar  world  and  the  impli¬ 
cations  of  new  problems  and  phenomena  in  that  world. 

Generally,  the  analytic  process  iterates  among  the  above  steps.  The 
problem  may  be  reformulated  after  looking  at  the  alternatives  or  syn- 
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thesizing  the  initial  results.  Interactions  with  the  decisionmakers 
may  lead  to  changes  in  the  objectives,  the  measures,  the  set  of  pos¬ 
sible  alternatives,  the  models,  and  expectations  about  the  future. 
The  role  of  the  model  in  this  framework  is  not  to  dominate  or  sup¬ 
plant  these  other  steps,  but  to  provide  the  means  for  quantifying  the 
effects  of  systems,  tactics,  and  strategies  on  measures,  given  the  sce¬ 
nario.  Thus,  it  glues  the  different  components  together  into  an  inte¬ 
grated  framework.  An  analysis,  particularly  of  military  operations, 
should  involve  many  computer  runs,  tests  of  hypotheses,  sensitivity 
tests,  “a  fortiori”  arguments,^  identification  of  cause  and  effect,  and 
in  general  an  elucidation  of  the  conditions  under  which  a  hypothesis 
holds  true  and  why.  In  fact,  a  good  analysis  can  overcome  many  of 
the  shortcomings  in  the  representational  aspects  of  a  model.  This  is 
one  of  our  major  themes — that  analysis,  particularly  in  addressing 
military  operations,  should  drive  model  requirements.  The  problem 
is  not  so  much  with  the  models  as  it  is  with  the  lack  of  recognition  of 
what  constitutes  good  model-based  analysis  and  the  conditions 
under  which  such  quantitative  analysis  is  appropriate.  Thus,  we 
provide  the  following  guiding  principles: 

The  analyst  plays  the  central  role.  The  analyst  is  central  to  the  pro¬ 
cess  because  he  or  she  discerns  what  the  analysis  process  implies  or 
does  not  imply  about  the  problem,  determines  the  set  of  experi¬ 
ments,  performs  the  tests  (involving  model  runs),  identifies  the 
cause-and-effect  relationships,  adjusts  the  model  or  hypotheses, 
overcomes  the  limitations  of  the  model,  integrates  results,  and  pro¬ 
vides  the  explanatory  and  educational  tutorial  about  the  results.  No 
model  that  we  know  of  provides  these  functions,  and  they  require 
much  more  than  the  mechanical  ability  to  develop  inputs  and  dis¬ 
play  outputs.  A  key  part  of  this  is  preparing  analysts  to  perform 
campaign  analysis  and  employ,  where  appropriate,  a  model.  It  is 
essential  to  know  when  the  analyst  is  adequately  prepared. 

The  analysis  leads,  not  the  model.  It  is  surprising  how  often  this 
principle  seems  to  be  violated.  Tbe  principle  is  that  one  should  first 
consider  what  has  to  be  done  from  an  analytic  viewpoint  (what  is  the 


Such  arguments  are  done  by  making  favorable  assumptions  about  a  policy  option.  If, 
under  these  assumptions,  the  option  does  not  pay  off,  it  is  unlikely  to  work  well  under 
less-favorable  assumptions.  Frequently,  these  favorable  assumptions  are  easier  to 
model  than  more  realistic  ones. 
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problem?  what  are  the  objectives?  what  are  the  measures?  how 
should  the  measures  be  calculated?)  and  then  choose  the  appro¬ 
priate  measurement  tools  or  models  within  that  context.  Too  often 
problems  are  approached  the  other  way  (what  can  we  use  our 
favorite  model  to  calculate?).  Too  often  an  analytic  question  is 
changed  to  adapt  to  a  model  rather  than  the  other  way  around.^ 

The  drive  for  detail  is  not  always  warranted  in  models  for  analysis. 
Over  the  years,  various  scientists  and  authors  have  also  suggested  the 
principle  we  advocate  here.  For  example,  Einstein  once  stated  that  a 
model  should  be  as  "simple  as  possible  but  no  simpler."  This  is  also 
implied  in  the  principle  of  Occam’s  Razot^  and  the  more  pedestrian 
variant  called  KISS  (Keep  It  Simple,  Stupid).  This  principle  recog¬ 
nizes  the  importance  of  the  ability  to  explain  results  and  that  unnec¬ 
essary  detail  in  a  model  will  hinder  the  ability  to  identify  cause  and 
effect.  These  are  often  in  direct  contradiction  to  the  inclination  of 
some  in  the  community  for  verisimilitude  in  models.  For  example, 
Newton’s  laws,  which  leave  out  the  effects  of  speed  on  mass,  are 
perfectly  adequate  for  the  analysis  of  most  land  and  air  vehicles.  A 
simple  model  that  lists  when  resources  become  available  in  the  the¬ 
ater  (rather  than  a  complete  strategic  mobility  model)  may  ade¬ 
quately  represent  the  deployment  of  different  units  when  more  pre¬ 
cisely  calculated  arrival  times  are  overwhelmed  by  uncertainties  or 
are  not  a  critical  factor  in  the  analysis.  And  unnecessary  attempts  to 
achieve  realism  can  easily  impede  an  analysis.  If  a  complete  strategic 
mobility  model  is  used  when  it  is  not  necessary,  the  setup  time  and 
data  requirements  would  unnecessarily  increase  the  cost  and  time  of 
the  analysis  without  really  making  it  better.  Conceivably,  the  cost  in 
time  could  reduce  the  number  of  trials  and  cases,  resulting  in  less- 
complete  analysis. 


2 

Given  this  principle,  an  important  issue  is  how  to  deal  with  the  difference  between 
the  relatively  short  time  in  which  an  analysis  must  be  done  and  the  relatively  long  time 
required  to  develop  a  specifically  tailored  model.  The  answer  seems  to  be  either  a 
single  multipurpose  model  that  can  deal  with  the  very  broad  issues  or  a  set  of  models 
(perhaps  within  a  single  modeling  environment)  capable  of  analysis  in  more  narrowly 
defined  policy  areas. 

3 

Non  sunt  multiplicanda  entia  praeter  necessitatem  [multiplicity  ought  not  to  be 
posited  without  necessity] — the  principle  of  Occam’s  Razor. 
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So  what  does  this  mean  for  a  campaign  model?  First,  we  need  to 
develop  a  better  understanding  of  warfare,  particularly  future  war¬ 
fare,  and  methods  for  representing  critical  interactions.  Second, 
since  the  level  of  detail  cannot  be  completely  predetermined,  some 
flexibility  in  representation  (multiple  resolution)  must  be  a  funda¬ 
mental  part  of  the  model  architecture,  along  with  the  ability  to  take 
out  completely  the  portions  that  are  irrelevant  to  the  analysis  at 
hand.  Flexibility  must  often  be  traded  off  against  simplicity  in  the 
same  way  that  the  breadth  of  analytic  requirements  must  be  traded 
off  against  model  development  constraints. 

Training  and  analysis  models  will  differ  from  each  other.  Although 
there  may  be  some  commonality  among  components,  training  and 
analysis  models  are  likely  to  diverge  in  what  they  are  trying  to 
accomplish  and  hence  how  and  what  they  represent.'*  Training 
models  must  present  a  reasonable  set  of  stimuli  and  responses  for 
the  trainee,  or  the  wrong  lessons  may  be  learned.  Thus,  verisimili¬ 
tude  for  specific  training  objectives  may  be  a  desirable  feature.  As  we 
have  just  noted,  this  verisimilitude  can  get  in  the  way  of  analysis. 

Models  of  combat  at  the  campaign  level  cannot  predict  outcomes  in 
the  strict  sense  of  other  scientific  experimentation.  There  are  too 
many  uncertainties  in  terms  of  human  interactions,  unknown  initial 
conditions,  unknown  combat  phenomenology,  and  unknown  effects 
that  prevent  strict  prediction  with  a  campaign  model.  In  the  end,  the 
model,  as  a  set  of  hypotheses  about  campaign  warfare,  can  only  be 
judged  to  be  plausible  and  to  be  consistent  with  our  best  views  of  the 
world  for  a  given  set  of  initial  conditions  and  assumptions.^  With 
these  assumptions  clearly  described,  the  model  can  be  used  to  help 
identify  and  analyze  the  relative  importance  of  various  systems, 
operational  concepts,  and  force  structures.® 


4 

For  a  much  more  detailed  discussion  of  the  differences  between  training  and  analy¬ 
ses  and  the  associated  models,  see  Dewar  et  al.  (1996). 

5 

A  view  of  the  world  may  be  a  very  informal  mental  model,  historical  experience, 
expert  opinion,  or  a  more-formal  set  of  models  and  data.  We  realize  that  there  will 
always  be  conflicting  views  of  the  future  world.  What  we  argue  for  here  is  a  rational 
consideration  of  plausible  variations  and  consistent  treatment  of  issues  across  these 
variations. 

®Even  if  a  model  is  not  strongly  predictive,  it  can  be  successfully  used  for  a  number  of 
significant  purposes.  See  Dewar  et  al.  (1996)  and  Hodges  and  Dewar  (1992). 
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Models,  no  matter  how  realistic,  will  not  replace  analysis.  Too  often 
in  the  current  environment,  an  analysis  is  simply  equated  with  one 
or  more  runs  of  a  comhat  model.  If  this  were  really  true,  the 
approach  of  adding  detail  to  models  and  removing  analysts  from  the 
loop  might  be  correct.  Such  “analyst-free”  models  have  actually 
been  suggested.  Based  on  our  previous  description  of  analysis,  this  is 
dangerously  wrong.  There  is  much  more  to  analysis  than  a  set  of 
computer  runs. 

The  analytic  process  should  depend  on  multiple  independent  models 
and  multiple  independent  analyses  of  the  same  problem.  The  cur¬ 
rent  emphasis  within  DoD  may  be  interpreted  as  an  attempt  to  con¬ 
solidate  the  number  of  models  with  a  goal  of  one  last  and  final 
supermodel  for  campaign  analysis.  If  combat  were  devoid  of 
unknowns  or  uncertainties,  or  at  least  were  replicable,  this  engineer- 
type  approach  might  well  be  preferred.  However,  we  believe  it  to  be 
unsuitable  for  campaign  analysis,  where  the  goal  is  more  to  learn  and 
explain  than  to  predict.  Comparative  analysis  often  highlights 
assumptions  about  data,  scenarios,  and  effectiveness  that  are  not 
apparent  even  in  multiple  trials  with  a  single  model.  Instead  of 
reducing  the  models  to  a  single  representation,  a  preferred  approach 
may  be  developing  methods  for  linking  diverse  analyses  to  a  com¬ 
mon  point  of  reference.  This  will  continue  to  allow  creativity  and  in¬ 
novation  at  the  same  time  as  it  allows  decisionmakers  to  be  able  to 
better  understand  the  context  of  a  given  analysis  and  integrate  across 
the  results  and  recommendations.  To  the  extent  that  DoD  efforts 
encourage  the  capability  to  include  multiple  models  (perhaps  inte¬ 
grated  within  an  architecture),  we  endorse  this  approach.^ 

Education  is  an  important  element  of  analysis.  Analysis  and  the  use 
of  models  should  educate,  not  merely  “sell,"  the  analyst  and  the 
decisionmaker.  If  the  “education”  doesn’t  occur,  we  have  probably 


^We  are,  however,  concerned  that  this  may  simply  be  yet  another  way  of  building  a 
supermodel  that  will  be  "all  things  to  all  people.”  Along  the  continuum  of  modeling 
architectures,  from  a  single  large  model  to  the  flexible  plug-and-play  environment,  are 
a  variety  of  approaches  that  have  been  experimented  rvith  in  the  past,  each  with  its 
corresponding  capabilities  and  limitations.  The  community  has  had  a  full  range  of 
experiences  with  single  large  models  (such  as  TACWAR  and  Thunder  at  one  end  of  the 
spectrum),  whereas  the  extreme  of  fully  flexible  architecture  is  far  less  tested  or 
proven. 
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failed  as  analysts.  The  education  process  needs  to  be  included  in  the 
analytic  plan. 

The  use  of  models  may  not  enhance  creativity  in  the  analytic  process. 
This  is  the  negative  side  of  the  use  of  models  in  analysis.  Because 
models  provide  a  limited  view  of  the  real  world  and  represent  only 
one  set  of  hypotheses  about  the  relationship  of  things,  they  do  not 
lend  themselves  to  the  development  of  new  hypotheses  or  the  exam¬ 
ination  of  effects  outside  their  scope.  Campaign  models  developed 
during  the  Cold  War  to  simulate  conflict  between  NATO  and  the 
Warsaw  Pact  did  not  generally  represent  the  maneuver  of  ground 
forces,  instead  focusing  on  attrition  warfare  as  the  “accepted”  man¬ 
ner  for  analyzing  the  war.  This  constraint  caused  analyses  of  the 
conventional  balance  to  emphasize  weapon  system  kill  potential 
rather  than  mobility,  flexibility,  and  information  systems.  To  over-, 
come  such  limitations,  people  must  be  involved.  Model  limitations 
may  be  overcome  through  the  use  of  games  (model  supported  or 
not),  brainstorming,  or  other  methods,  but  it  is  generally  foolish  to 
expect  a  model  itself  to  provide  much  in  the  way  of  creative  insights 
about  a  problem  or  its  solution. 

MANAGING  THE  DEVELOPMENT  PROCESS 

The  development  of  a  new  campaign  model  or  the  making  of  signifi¬ 
cant  improvements  to  an  existing  model  is  not  a  light  undertaking. 
Managing  the  process  includes  not  only  configuration  management 
of  the  code,  but  also  (1)  documenting  the  overall  rationale  for  the 
model,  (2)  establishing  requirements  and  methods  for  calculating 
measures  and  representing  objects,  (3)  using  resources  successfully 
(people,  time,  and  money),  and  (4)  and  dealing  with  organizational 
constraints.  Generally,  there  is  much  more  to  the  development  than 
expected;  because  much  about  the  combat  phenomenology  is  not 
understood  well  enough  to  codify,  there  is  considerable  risk  and 
uncertainty  that  the  final  outcome  will  only  be  a  technical,  not  a 
substantive,  improvement  over  what  already  exists.  Too  often, 
unreal  expectations  and  an  unconditional  “can  do”  attitude  can  lead 
to  development  plans  and  decisions  not  seasoned  with  the  wisdom 
and  caution  of  experience.  This  can  spell  disaster  on  the  model- 
development  trail. 


Applying  a  Campaign  Model  to  Analysis  13 


There  are  many  reasons  to  develop  new  models,  not  all  of  which 
have  to  do  with  analytic  needs.  Motivating  factors  range  from  the 
desire  to  take  advantage  of  new  computer  hardware  and  software  to 
the  creative  fun  of  developing  something  new.  Perhaps  the  most 
important  reason  is  to  better  represent  phenomenology  hitherto 
omitted  in  models  and  to  create  better  methods  for  exploring  uncer¬ 
tainties  and  unknowns.  There  are  also  strong  disincentives  in  the 
form  of  cost,  training  requirements,  organizational  inertia,  and  sus¬ 
tainability.  Too  often,  however,  promises  based  on  unwritten  soft¬ 
ware  and  considerable  underestimates  of  the  costs  and  development 
time  required  lead  to  the  premature  initiation  of  a  new  model  devel¬ 
opment.®  Although  we  heartily  endorse  an  ongoing  process  of  model¬ 
ing  as  a  means  of  improving  understanding  and  codifying  what  we 
learn  about  combat,  the  decision  to  develop  a  new  campaign  model 
for  analysis  should  be  approached  with  considerable  logic  and  plan¬ 
ning  and  should  be  as  independent  as  possible  of  the  organizational 
competition  of  the  various  DoD  analysis  agencies,  services,  consul¬ 
tants,  and  contractors. 

A  series  of  questions  about  the  rationale  for  a  new  model  should  be 
asked.  These  include 

1 .  What  is  the  range  of  analytic  needs  for  the  model? 

2.  Will  the  new  model  simulate  phenomenology  not  represented  in 
existing  models? 

3.  Can  we  do  a  significantly  better  job  at  representing  those  phe¬ 
nomena  already  included  in  existing  models? 

4.  How  does  the  development  schedule  compare  to  the  historical 
experience  in  developing  such  campaign  models? 

5.  How  precisely  are  the  design  requirements  defined? 

6.  Who  is  on  the  development  team,  and  what  is  the  members’ 
experience  in  producing  this  type  of  model? 

7.  What  are  the  near-term  expectations  of  decisionmakers? 


g 

We  note  that  many  new  modeling  developments  are  associated  with  an  overall 
architecture  or  environment  to  facilitate  the  coding,  use,  and  maintenance  of  the 
model  or  models.  Our  recommendations  apply  to  these  development  efforts  as  well. 
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8.  Has  there  been  a  study  of  the  existing  models  to  determine  if 
they  can  meet  the  need  or  if  they  can  be  more  easily  modified  to 
accomplish  the  goals  than  trying  to  build  a  new  model? 

9.  Does  the  new  model  have  the  support  of  the  various 
organizations  that  are  expected  to  use  it? 

10.  What  constraints  on  cost,  people,  and  time  resources  will  apply 
to  the  use  of  the  model? 

Given  satisfactory  answers  and  a  decision  to  develop  a  model,  many 
aspects  of  the  design  and  development  of  a  new  model  need  to  be 
understood,  including  the  establishment  of  functional  requirements, 
defining  the  underlying  structure  of  a  campaign  model,  creating  real¬ 
istic  expectations,  prototyping,  testing,  object- system  modeling, 
choosing  the  level  or  levels  of  aggregation,  defining  the  operating 
environment,  determining  the  data  requirements,  and  phasing  in  the 
new  model. 

One  key  decision  to  be  made  in  the  development  process  is  how 
submodels  will  be  linked  to  the  simulation.  A  variety  of  approaches 
can  be  imagined — creating  one  big  model,  confederating  the  models 
with  a  model-specific  interface,  linking  the  models  with  a  general¬ 
ized  set  of  communication  protocols,  designing  an  overall  architec¬ 
ture  or  environment  in  which  models  can  be  "plugged-and-played,” 
and  establishing  a  loose-knit  hierarchy  or  family  of  models.  Each  of 
these  methods  has  advantages  and  disadvantages  in  terms  of  main¬ 
taining  the  appropriate  conceptual  linkage  between  the  models, 
achieving  model  flexibility,  and  demonstrating  operational  feasibil¬ 
ity.  The  purpose  of  the  model  and  how  the  simulation  development 
is  managed  will  determine  in  large  measure  which  approach  should 
be  used. 

Ten  of  the  most  important  pitfalls  that  can  be  avoided  with  appro¬ 
priate  planning  are 

1.  not  involving  the  modelers  and/or  the  prospective  analysts  in 
the  entire  design  process 

2.  premature  attempts  at  analysis  with  a  new  model 
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3.  not  performing  early  tests  of  both  model  components  and  the 
full  model  with  realistic  problems® 

4.  unrealistic  schedules  and  attempts  to  satisfy  an  unrealistically 
diverse  range  of  requirements 

5.  not  knowing  what  is  hard,  what  is  easy,  and  what  we  do  not 
know  how  to  do 

6.  not  establishing  organizational  and  institutional  issues  and 
influences  on  the  development 

7.  not  considering  sources  of  data  early  in  the  development 

8.  defining  requirements  independent  of  the  analytical  framework 
and  resource  constraints 

9.  not  spending  enough  time  learning  what  has  been  done  in 
previous  models 

10.  believing  too  strongly  that  new  software,  modeling,  or  data 
paradigms  will  make  all  the  hard  problems  easier. 


OVERCOMING  PROBLEMS  OF  DATA  AND  INTEGRATING 
WITH  OTHER  MODELS 

The  data  are  often  more  important  than  the  model.  Because  of  the 
quantity  and  breadth  of  data  required,  it  takes  a  great  deal  of  care  not 
to  have  errors  in  the  input  data  of  a  campaign  model.  There  are 
other  reasons  that  data  tend  to  dominate  modeling  in  performing 
analysis.  Data  preparation  is  often  the  most  time-consuming  and 
resource-intensive  aspect  of  the  analysis.  Given  the  time  urgency  of 
many  analyses,  there  are  many  opportunities  to  make  errors,  and 
resource  limitations  cause  pressure  to  take  shortcuts  by  using  old  or 
inappropriate  data  sets.  Data  become  even  more  important  in  the 
current  trend  to  define  more  and  more  of  the  object  representations, 
processes,  and  logic  of  a  model  as  data.  The  distinction  between 
model  code  and  data  is  increasingly  blurred  with  modern  computing 
languages. 


®Items  2  and  3  directly  oppose  each  other.  Pushing  the  model  into  production  too 
soon  can  be  very  disruptive.  However,  not  demonstrating  capabilities  early  can  be 
fatal. 
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Database  structures  and  processes  interact  with  the  model.  Once  the 
databases  and  input  stmctures  and  processes  are  created,  changes  to 
the  model,  such  as  objects  in  the  model,  will  likely  require  different 
data  or  different  aggregations  of  the  data.  This  will  require  changes 
to  the  database  structure,  including  new  or  modified  input  files, 
database  processes,  and  new  input  processes  within  the  model  itself. 
It  may  also  call  for  changes  in  the  graphical  user  interfaces  and  any 
preprocessors  for  the  input  files.  Thus,  model  changes  can  ripple 
through  all  of  the  input-associated  processes  and  the  organization  of 
the  input  databases.  The  unfortunate  result  is  that  these  aspects  of 
the  input  processing  create  very  real  constraints  on  how  the  model 
can  be  changed  and  inhibit  the  flexibility  of  the  model.  An  important 
element  of  the  design  of  a  flexible  model  is  to  create  a  set  of  flexible 
input  tools  and  model,  data,  and  coding  structures. 

Data  input  and  output  processes  can  greatly  enhance  a  model  and 
its  usability.  The  data  handling  in  modern  campaign  models  can  be 
assisted  greatly  by  various  input  and  output  processes  and  stan¬ 
dards.  Graphical  user  interface  methods  to  limit-check  and  cross¬ 
check  data,  data  dictionaries,  database  manipulation  functions  (such 
as  aggregations),  and  data  output  processes  (such  as  visualizations) 
all  reduce  the  work  of  data  preparation  and  assist  in  the  important 
checking  and  verification  of  the  data. 

Much  of  the  data  must  be  developed  from  detailed  (higher- 
resolution)  models.  Because  of  the  lack  of  any  other  data  source 
(history,  training,  tests,  or  even  expert  opinion),  the  sources  of  many 
data  items  for  a  campaign  model  are  the  computations  of  other, 
higher-resolution  models.  Historical  data,  for  example,  are  typically 
not  for  the  right  situations,  not  well  documented,  or  of  such  an  ag¬ 
gregate  nature  that  they  can  only  be  used  as  a  benchmark  for  the 
data  the  analysis  requires.  This  is  not  to  say  that  data  from  more- 
detailed  models  are  unconditionally  better  than  these  other  sources, 
but  rather  that  their  use  is  an  alternative  means  that  is  often  turned 
to  as  the  preferred  approach.  The  process  of  “tuning"  to  a  higher- 
resolution  model  is  an  art  and  requires  a  knowledgeable  analyst  to 
determine  how  to  aggregate  the  data,  when  the  approximation  is 
good  enough,  what  cases  to  use  from  the  high-resolution  model,  and 
which  parameters  to  adjust  to  achieve  a  good  approximation. 
Aggregation  can  be  done  against  time,  geographic  dispersion,  types 
of  objects,  or  measures.  The  approach,  data,  and  tests  of  the 
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approximations  should  be  published  and  subject  to  peer  review  to 
help  establish  credibility  and  consistency.  Unfortunately,  very  few  of 
these  approaches  have  been  openly  described  or  performed  in  this 
manner. 

Consolidate  data-production  efforts.  Our  final  point  about  data  is 
that  a  common  effort  by  the  defense  community  could  reap  dramatic 
benefits.  Such  an  effort  would  involve  sharing  data,  critiquing  ap¬ 
proaches  for  data  generation,  and  testing  different  approaches  to 
linking  models  and  aggregating  data.  In  the  current  environment, 
there  is  too  much  redundant  effort  at  developing  databases  and  too 
little  testing  and  review  of  approaches,  including  selection  of 
sources,  embedded  assumptions,  and  appropriate  usages.  Both  the 
quality  and  timeliness  of  analysis  suffer  from  the  current  organiza¬ 
tional  barriers  that  unnecessarily  impede  data  access  and  lack  of 
scientific  approaches  to  data  production  across  the  wide  variety  of 
potential  sources. 

CREATING  FLEXIBLE  UNDERLYING  STRUCTURES 

By  the  underlying  structures  of  a  campaign  model,  we  mean  the 
treatment  of  simulated  time  and  events,  the  interfaces  with  the  ana¬ 
lysts,  the  representation  of  spatial  objects,  the  approach  to 
representing  command  and  control,  the  simulation  of  inherently 
random  or  uncertain  events,  and  the  type  of  software  construction 
used.  The  choice  of  structure,  one  of  the  most  important  decisions 
with  respect  to  any  model  development,  determines  how  objects  and 
information  can  be  aggregated  and  what  the  basic  level  of  resolution 
or  potential  for  multiple  resolution  the  simulation  can  represent. 
The  structure  affects  the  adaptability  of  the  simulation  to  a  range  of 
analytic  problems,  the  efficiency  in  processing,  the  transparency  of 
cause-and-effect  relationships,  and  the  ease  with  which  future 
changes  can  be  made.  These  structural  choices  must  be  made  early 
in  the  design  phase  of  a  model  and  subsequently  affect  all  other 
aspects  of  model  development.  They  must  be  resolved  in  the  context 
of  the  analytic  requirements  and  development  constraints. 

To  understand  the  effects  of  structural  choices,  one  need  look  no 
further  than  many  of  the  current  campaign  models.  The  Cold  War 
spawned  a  set  of  ground-warfare  campaign  models  that  represent 
ground  operations  as  a  set  of  “pistonlike”  movements.  Many  of  these 
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models  are  still  in  use  and  perhaps  appropriate  for  some  applica¬ 
tions,  but  constrain  the  ability  to  simulate  maneuver  warfare  so  vital 
to  representing  combat  in  some  regions  of  the  world.  In  a  similar 
vein,  many  of  the  early  representations  of  theater  air  operations  used 
“time-stepped”  models  of  air  and  ground  operations  with  relatively 
large  time  steps — a  few  hours  to  a  day.  White  perhaps  adequate  for 
large-scale,  doctrinal  air  operations,  these  models  limit  the  ability  to 
look  at  timing  and  command  and  control  issues  associated  with  both 
friendly  and  threat  systems  employing  new  capabilities  (such  as 
onboard  and  off-board  sensor  systems)  and  new  concepts  (such  as 
reactive  maneuver). 

It  is  not  so  much  that  specific  structural  choices  are  inherently  bad 
but  that  their  limitations  should  be  explicitly  identified  and  under¬ 
stood  when  the  choice  is  made.  Furthermore,  some  structures  are 
less  limiting  than  others  in  terms  of  campaign-level  representation. 
For  example,  flexible  network  structures  can  represent  both  piston 
movement  and  nonlinear  maneuvers.  Decision  tables  and  goal- 
seeking  algorithms  can  be  used  in  place  of  scripts,  but  not  vice  versa. 
The  tradeoff  between  flexibility  and  the  cost  to  develop,  use,  and 
maintain  a  particular  structure  must  be  wisely  considered.*® 

The  components  of  structure  of  a  campaign  simulation  that  we 
address  in  detail  in  the  main  report  include 

1.  simulation  time  management — whether  time  is  advanced  in 
event  steps,  fixed  time  steps,  or  continuously 

2.  spatial  structure — the  representation  of  movement,  object  posi¬ 
tions,  and  terrain 

3.  stochastic  structure — random  or  deterministic 

4.  software  structure — the  software  paradigm,  such  as  object- 
oriented  coding,  structured  programming,  and  process-oriented 
programming 

5.  distribution  structure — distributed  processes,  distributed  trials, 
single  processor,  and  parallel  processing 


Cost  is  used  very  broadly  here  to  include  monetary  cost,  time,  and  skill  levels  that 
must  be  expended.  Flexibility  and  cost  are  typically  directly  opposed  to  each  other. 
More  is  said  about  specific  options  in  the  main  report. 
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6.  human  interaction  and  command  and  control  structure — closed, 
open,  scripted,  human-in-the-loop,  semiautomated  planning, 
and  decision  table 

7.  resolution  structure — fixed,  variable,  and  selectable  resolution. 

REPRESENTING  DECISIONMAKING  AND  COMMAND, 
CONTROL,  COMMUNICATIONS,  COMPUTERS,  AND 
INFORMATION  (CU) 

Decisionmaking  and  C^I  issues  dominate  much  of  campaign  analy¬ 
sis.  At  the  campaign  level,  the  problem  to  be  addressed  is  often  not 
how  much  force  there  is,  but  how  it  is  used.  Despite  the  importance 
of  this  problem,  it  remains  the  least  understood  and  most  inade¬ 
quately  modeled  part  of  most  campaign-level  simulations.  In  some 
models,  nearly  perfect  battlefield  intelligence  is  simulated,  and  each 
side  has  unrealistically  perfect  knowledge  of  the  other  side’s  posi¬ 
tions,  forces,  capabilities,  and  intents.  Furthermore,  nearly  instanta¬ 
neous  planning  and  execution  are  implicitly  represented  by  many  of 
the  models  and  in  effect  assume  a  nearly  perfect  command  and  con¬ 
trol  process.  Many  combat  phenomena  are  less  than  adequately 
treated  in  the  current  generation  of  combat  models.  We  believe  C^I 
is  the  one  phenomenon  most  urgently  in  need  of  improvement. 

It  is  believed  by  many  that  the  U.S.  dominance  of  the  information 
assets  in  a  campaign  should  provide  a  distinct  advantage  over  most 
of  our  potential  opponents.  The  ability  to  view  opposing  forces  and 
maneuvers,  to  disrupt  planning  and  execution,  and  to  feed  misin¬ 
formation  while,  at  the  same,  time  denying  the  enemy  the  same 
capabilities  against  us  should  alter  the  character  of  the  war  and  pro¬ 
vide  U.S.  and  friendly  forces  with  a  distinct  advantage.  Campaign 
models  should  show  these  interactions  and  help  analysts  determine 
the  value  of  acquiring  various  C^I  assets. 

The  starting  point  for  understanding  C'*!  in  a  campaign  model  is  to 
look  at  the  decisions  and  force  allocations  that  depend  on  the  effec¬ 
tiveness  of  C^I.  At  the  highest  level,  there  are  campaign  objectives 
and  the  commander-in-chief  s  strategies  for  achieving  these  objec¬ 
tives  within  the  constraints  of  the  campaign.  These  include  the 
phasing  of  the  campaign  and  the  emphasis  on  broad  operational 
objectives:  targets  destroyed,  air  superiority,  and  territory  won  or 
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lost.  At  the  next  tier,  the  operational  level,  are  the  decisions  about 
force  allocations,  priorities,  and  timing  needed  to  achieve  opera¬ 
tional  objectives.  These  include  how  multirole  aircraft  are  allocated 
to  missions  over  time,  how  the  ground  forces  are  reinforced,  and 
what  types  of  targets  to  emphasize  over  time.  We  group  the  deci¬ 
sions  at  the  third  level  as  tactical  taskings.  These  include  the  alloca¬ 
tion  of  weapon  fires  of  a  ground  unit  against  the  weapons  and  forces 
of  the  other  side,  the  allocation  of  fighters  against  penetrating 
bombers  and  escorts,  the  choices  about  which  airfields  or  strategic 
targets  to  attack,  and  the  allocation  of  surface-to-air  fire  of  a  coordi¬ 
nated  (netted)  set  of  surface-to-air  missile  fire  units. 

In  general,  it  is  difficult  at  the  campaign  level  to  provide  a  detailed 
representation  of  all  of  the  information-gathering  objects,  the  com¬ 
munications,  and  all  levels  of  command  and  control.  The  problem, 
then,  is  to  show  the  essence  of  C^I  in  a  campaign  and  to  determine 
how  to  model  and  analyze  its  effects.  We  next  examine  some 
approaches  to  capturing  these  effects  more  fully. 

Many  models  use  scripted  force  allocations  as  inputs  to  the  simula¬ 
tion.  These  scripts  do  not  adapt  to  changes  in  situation  during  the 
course  of  the  simulated  battle.  Most  models  do  not  adequately 
reflect  the  competitive  nature  of  decisionmaking  between  opponents 
on  a  battlefield.  Thus,  a  great  deal  of  manual  intervention,  planning, 
and  replanning  of  force  allocations  is  required.  We  desire  to  find 
alternative  means  that  automate  some  of  these  processes.  In  some 
cases,  we  may  wish  to  simulate  the  behavior  of  human  decisionmak¬ 
ers.  However,  in  many  analyses,  it  is  important  to  have  the  best  pos¬ 
sible  decisions  made,  according  to  doctrine  or  based  on  some  simple 
rules  of  thumb  that  might  include  cultural  biases.  The  method 
depends  on  the  application.  Thus,  what  we  are  arguing  for  is  the 
ability  to  represent  the  decisionmaking  process  in  a  variety  of  differ¬ 
ent  ways  and  to  be  able  to  select  the  method  that  best  suits  the 
analytic  need. 

We  compare  some  of  the  approaches  RAND  and  others  have  tried 
and  describe  some  of  the  aspects  of  command  and  control  phe¬ 
nomenology  that  are  not  yet  well  enough  understood  to  include  in  a 
campaign  model.  The  approaches  include: 
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Interactive  or  human-in-the-loop.  These  simulations  allow  for 
human  beings  to  interact  with  the  flow  of  the  simulation — not 
just  at  the  beginning  of  the  run  but  throughout. 

Scripted  decisions.  This  method  does  not  rely  on  human  inter¬ 
action  throughout  the  simulation.  What  happens  and  when  are 
specified  up  front  with  consistency  and  repeatability.  An  alter¬ 
native  technique  to  obtaining  a  script  is  to  execute  the  model 
iteratively.  At  each  decision  point,  the  option  is  chosen  that  is 
viewed  as  best  in  light  of  what  has  been  learned  from  previous 
runs  of  the  model. 

Rule-based  or  expert  systems.  To  introduce  adaptability  into 
scripts  and  mimic  doctrinal  guidance,  one  may  instead  adopt  a 
set  of  rules.  The  rules  are  sometimes  organized  into  decision 
tables.  Decision  tables  list  the  conditions  and,  for  each  condi¬ 
tion,  the  corresponding  decision  to  be  taken. 

Tactical  algorithms.  Several  lower-level,  yet  still  important, 
decisionmaking  processes  may  be  implemented  directly  in  code. 
These  tend  to  be  more-complex  extensions  of  rule-based  sys¬ 
tems. 

Value-driven  method.  The  outcome  state  of  each  possible  deci¬ 
sion  is  estimated  or  predicted,  perhaps  with  a  “look-ahead” 
method.  The  outcome  state  is  evaluated  according  to  some 
metric,  and  the  action  that  incrementally  optimizes  this  function 
is  selected. 

Learning  algorithms.  This  extension  of  value-driven  methods 
allows  the  simulated  decisionmaking  process  to  learn.  A  nu¬ 
meric  “fingerprint”  of  the  current  state  of  the  system  is  retained 
whenever  it  makes  a  decision.  Future  decisions,  although  un¬ 
likely  to  be  based  on  exactly  the  same  circumstances,  examine 
previous  fingerprints  and  the  desirability  of  the  outcome.  When 
circumstances  exist  beyond  the  reasonable  bounds  of  previous 
decisions,  other  methods  are  used  to  determine  the  actions. 

Objective -driven  optimization  and  gaming  methods.  Opti¬ 
mization  techniques  are  used  to  search  the  decision  space 
efficiently.  The  function  that  determines  the  value  of  the  deci¬ 
sion  is  called  the  objective  function,  and  the  principles  of  math¬ 
ematical  programming  are  brought  to  bear  to  maximize  (or 
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minimize)  an  objective  function  efficiently.  Some  of  the  meth¬ 
ods  employed  involve  linear  programming,  shortest  path,  maxi¬ 
mum  flow  algorithms,  or  other  search  techniques  (such  as 
genetic  algorithms). 

These  various  approaches  to  the  decisionmaking  process  have 
capabilities  and  limitations  related  to  their  ease  of  implementation, 
ease  of  use,  ability  to  be  adaptive,  ability  to  represent  specific  deci¬ 
sionmaking  processes  better,  and  inclusion  of  natural  biases. 

ACHIEVING  TRANSPARENCY  IN  THE  USE  OF  A  CAMPAIGN 
MODEL 

One  of  the  key  challenges  of  modeling  for  analysis,  especially  vnth 
large  campaign  models,  is  to  understand  clearly  the  cause-and-effect 
relationships  that  occur  within  the  model.  Outcomes  may  result 
from  new  systems,  strategies,  assumptions,  effectiveness  inputs,  or 
even  errors  in  the  model  or  data.  Rather  than  just  stating  model 
results,  the  analyst  carries  the  responsibility  of  identifying  how 
changes  in  the  inputs  affect  output  measures  and  how  these  are  in 
turn  affected  by  other  data  or  assumptions.  Because  a  campaign 
model  has  a  formidable  quantity  of  data  representing  the  forces, 
effectiveness,  scenario,  and  setting,  it  is  not  generally  feasible  to  vary 
all  combinations  of  the  input  data. 

Most  analysts  realize  that  it  is  not  enough  to  present  a  set  of  out¬ 
comes  from  a  campaign  model.  Furthermore,  the  analyst  should  be 
able  to  motivate  and  argue  model  results  without  reference  to  the 
model.  Along  with  the  outcomes,  one  must  describe  the  assump¬ 
tions,  describe  the  data  used,  and — most  importantly — provide  the 
logic  to  explain  why  the  results  are  as  they  are.  The  ability  to  explain 
the  results  depends  on  transparency.  This  in  turn  depends  on  both 
the  analytic  process  and  the  analyst’s  understanding  of  the  relation¬ 
ships  in  the  campaign  model — explanation  of  the  results  is  only  par¬ 
tially  a  function  of  the  model.  Certainly  a  very  simple  model  is  easier 
to  understand  than  a  complex  model,  but  a  model  with  only  a  few 
equations  or  relationships  can  also  appear  complicated.  Simple 
stochastic  equations  are  not  so  simple  to  someone  not  trained  in 
probability  theory.  Understanding  depends  on  the  education, 
training,  and  experience  of  the  analyst  using  the  model.  Even  when 
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the  model  is  understood,  the  complex  relationship  of  the  outputs  to 
the  potentially  large  number  of  inputs  and  assumptions  of  a  cam¬ 
paign  model  requires  an  analytic  process  (which  is  usually  not  part  of 
the  model)  to  identify  the  causes  and  effects  in  a  study.  This  process 
is  important  for  separating  erroneous  or  misleading  results  from 
those  that  are  truly  plausible  and  justifiable. 

The  most  important  thing  to  understand  is  that  transparency  is  not 
an  inherent  attribute  of  any  model.  Software  features  typically 
thought  of  to  enhance  the  transparency  of  the  model  include 

•  summary  or  aggregated  outputs 

•  map  graphic  outputs 

•  error-,  limit-,  and  cross-checking  of  inputs 

•  graphical  user  interface 

•  automated  sensitivity  testing. 

We  do  not  mean  to  minimize  in  any  way  the  great  contribution  these 
computing  and  data-processing  capabilities  can  make  to  the  under- 
standability  of  the  model.  Too  often,  however,  little  or  no  attention  is 
given  to  the  analytic  processes  mentioned  above.  Even  the  physical 
model  relating  force  to  mass  and  acceleration  (F  =  ma)  is  probably 
not  understood  by  a  large  segment  of  the  general  population. 
Contrary  to  beliefs  still  held  in  the  community,  transparency  is  usu¬ 
ally  not  obtained  by  increasing  the  detail  or  verisimilitude  of  the 
model.  Increasing  the  apparent  “realism”  increases  complexity  and 
often  adds  more  detail  than  is  necessary  for  many  applications  of  the 
model. 

In  the  main  report,  we  discuss  more  thoroughly  how  transparency 
can  be  achieved  by 

•  The  analytic  process.  The  first  step  is  to  define  the  analytic 
approach  to  the  problem  in  a  way  that  helps  to  provide  an 
understanding  of  objectives,  measures,  constraints,  and  alterna¬ 
tives,  as  well  as  of  the  explanatory  logic  for  observations  and  rec¬ 
ommendations. 

•  Hierarchical  sensitivity  testing.  Classical  sensitivity  analysis 
examines  how  changes  in  inputs  affect  outputs.  Typically,  the 
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assumptions  and  input  changes  are  too  numerous  to  examine 
exhaustively.  One  common  approach  is  to  apply  some  kind  of 
generalized  experimental  design  or  response-surface  methodol¬ 
ogy.  However,  for  campaign  analysis,  it  is  soon  realized  that 
these  traditional  approaches  work  only  for  simple  problems  and 
fail  when  the  number  of  variables  gets  large.  It  is  therefore  more 
appropriate  to  adopt  a  hierarchical  approach  that  relies  on  some 
knowledge  of  the  problem,  limiting  the  space  of  sensitivities  that 
must  be  calculated. 

•  Comparative  analysis.  Here,  the  idea  is  to  learn  through 
understanding  the  differences  in  similar  analyses.  Although  this 
process  requires  a  good  deal  of  effort  (perhaps  more  than  is 
affordable),  comparisons  of  this  kind  have  provided  greater 
insight  and  transparency  into  the  models  and  assumptions  used. 
This  is  generally  done  with  different  models,  although  a  variation 
could  involve  a  different  group  using  the  same  model  and 
developing  at  least  some  of  its  own  data  and  assumptions. 

•  Education  and  training.  It  should  be  obvious  that  an  analyst 
who  really  understands  a  model  will  be  better  able  to  explain  its 
results  than  someone  just  learning.  What  may  not  be  obvious  is 
that,  for  most  campaign  models,  this  understanding  takes  a  long 
time,  perhaps  years,  and  it  takes  more  than  just  introductory 
training  with  the  model  for  military  operations. 

In  the  end,  trust  in  the  model,  analyst,  and  data  will  provide  some 
comfort  that  mistakes  have  been  avoided,  but  it  will  not  substitute 
for  the  explanatory  transparency  developed  by  the  analytic  process. 


Chapter  Three 

CONCLUSIONS:  A  PROGRAM  FOR  THE  FUTU^ 


Future  model-based  campaign  analysis  depends  on  the  resolution  of 
a  number  of  issues.  We  have  emphasized  the  importance  of  focusing 
on  analysis  rather  than  model  verisimilitude  as  the  means  of 
significantly  advancing  campaign  analysis.  It  is  necessary  for  the 
community  to  address  some  of  the  key  data  issues  impeding  cam¬ 
paign  analysis,  primarily  by  better  cooperating  in  data  production 
and  sharing  and  by  opening  analyses  up  for  greater  community 
review.  Certainly,  the  models  should  be  improved  in  terms  of  more- 
flexible  structures,  variations  in  resolution,  better  representation  of 
combat  phenomena  (particularly  C^I),  and  greater  transparency. 
This  final  section  summarizes  some  of  the  actions  that  the  defense 
modeling  and  analysis  community  should  take  to  move  campaign 
analysis  into  the  next  generation.  These  actions  are  explained  in 
more  detail  in  the  last  section  of  the  main  report: 

1.  Educate  analysts  and  decisionmakers  about  the  needs,  methods, 
and  limitations  of  model-based  campaign  analysis. 

2.  Balance  the  emphasis  on  models  with  an  emphasis  on  improving 
analysis,  particularly  defining  objectives  and  measures,  building 
the  rationale  for  conclusions,  and  better  informing  decision¬ 
makers. 

3.  Improve  database  development  and  sharing  within  the  defense 
community. 

4.  Design  and  develop  a  set  of  campaign  models  with  a  range  of 
capabilities. 
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5.  Focus  research  and  development  on  the  effects  and  representa¬ 
tion  of  key  combat  phenomena. 

6.  Adopt  a  more  “scientific”  approach  to  analysis,  including  critical 
peer  review  and  broader  disclosure  of  methods  and  results. 

Ongoing  DoD  initiatives,  particularly  the  Joint  Analytic  Modeling 
Improvement  Program  (JAMIP),  are  trying  to  address  many  of  these 
issues.  This  and  the  main  report  provide  specific  suggestions  that  we 
believe  are  necessary  for  these  efforts  to  achieve  next-generation 
capability  to  support  analysis. 

In  conjunction  with  this,  the  authors  applaud  the  DoD’s  current 
review  of  campaign  modeling,  and  recommend  that  a  heavier 
emphasis  be  placed  on  the  approach  and  requirements  for  cam¬ 
paign-level  analysis.  A  similar  review  of  analysis  could  summarize 
the  activities  of  current  DoD  analytic  organizations  and  estimate  the 
future  needs  for  analysis,  evaluate  the  approaches  currently  being 
used  in  analysis,  evaluate  the  use  of  campaign  models  in  analysis, 
consider  the  effects  of  new  models  and  Advanced  Distributed 
Simulation  on  analysis,  and  compare  the  projected  needs  for  analyti¬ 
cal  staff.  The  review  should  be  done  by  a  committee  composed  of 
experts,  users,  and  practitioners  of  analysis.  It  should  be  oriented 
not  solely  toward  cost  savings  or  efficiency  but  toward  improving  the 
analytic  capabilities  and  products  of  the  defense  community.  It 
should  not  focus  solely  on  the  models  used  for  analysis.  Rather,  it 
should  consider  the  questions  being  asked,  the  analytical  ap¬ 
proaches,  the  management  of  data,  the  resource  constraints  to 
analysis,  the  role  of  models  in  defense  analysis,  the  products  of  the 
analysis,  and  the  influence  of  those  products  on  defense  policy 
decisions. 
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